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Networks 


Networks  ...  Connect 


Telecommunications  Networks,  Traffic,  &  Engineering 


Networks: 

Tele-Traffic: 

Engineering: 


A  Deep  Hierarchy  of  Systems 

A  Turbulent  River  of  Myriad  Data  Sources 

Traffic  over  a  Network:  Designing,  Managing,  Optimising 


Networks:  A  Deep  Hierarchy  of  Systems 


•  Connectivity  (node  organisation,  placement,  topology). 

•  Physical  layer  technology  (devices,  timing,  coding,  error  recovery..). 

•  Circuit  vs  Packet  Switched  Paradigms. 

•  Connection-oriented  vs  connectionless  philosophy. 

•  Protocol  hierarchy  and  encapsulation,  eg:  Ethernet  IP  [  TCP[HTTP]  ]  ]. 

•  Operation  (routing,  signaling,  admission  control,  congestion  control). 

•  Inter-Networking  (Autonomous  systems,  domain  routing,  addressing,  gate¬ 
ways,  protocol  translation). 
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Tele-Traffic:  A  Turbulent  River  of  Myriad  Data  Sources 

•  ‘Geographic’  Complexity 

•  At  network  edge  (distribution  of  sources,  destinations,  nodes) 

•  Internally  (multiplexing  and  de-multiplexing  of  streams) 

•  Offered  Traffic  Complexity 

•  User ‘sessions’  (durations,  arrivals,  number  of  clicks..) 

•  Applications  used  (browser,  napster..) 

•  Underlying  protocols  (TCP,  HTTP.) 

•  Underlying  data  objects  (files,  video,  audio..) 

•  Shaping  by  network  elements 

•  Temporal  Complexity: 

•  Human  driven  (diurnal,  coffee  breaks,  think  times..) 

•  Source  driven  (real-time  constraints,  file  sizes..) 

•  Network  driven  (topology,  protocols..) 

•  Technology  driven  (link  rates,  switching  rates) 

->  Time  scale  rich:  ns  to  months,  1  kbps  to  Terabits/s 
->  Burstiness:  temporal  (scaling),  amplitude  (non-G),  spatial  (capacity  diversity) 
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TeleTraffic  Engineering:  the  traditional  program 


•  Collect  traffic  measurements  and  measure  characteristics 

•  Select  traffic  model 

•  Solve  switch  performance  problems 

•  Solve  network  performance  problems 

•  Design  network  to  given  quality  of  service  at  minimal  cost 

•  Solve  admission  control  problems 

•  Run  network 

•  Solve  congestion  control  and  routing  problems 

•  Improve  network  performance 
-  Iterate  until  paradigm  shift 

This  ‘scientific  method’  approach  begins  with  measurements... 
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Traffic  Modelling  in  1988 


Papers  between  1 966  and  1 987  (p.  f.  pawnta,  uc-12,  itaiy) 


•  Queueing  theory:  several  thousand 

•  Traffic  measurement:  around  50. 
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Tele-Traffic  Today 


Measurement  Practice 

•  Recognition  of  measurement,  not  just  routine  quantification,  but  discovery. 

•  Widespread  monitoring  of  LAN’s,  ISP  networks,  national  infrastructures . 

•  Large  scale  programs  for  Internet,  from  routine  monitoring  to  ultra  high  resolution. 

•  Emergence  of  numerous  small  &  large  scale  active  probing  efforts. 

•  Huge  advances  in  measurement  accuracy. 


Renaissance  in  Modelling 

•  Return  of  science  101 :  Observation  and  inspiration  before  model. 

•  Return  of  verification:  Evaluating  usefulness  against  real  data. 

•  New  model  paradigm  as  standard:  Characteristic  scale  — ►  scale  invariance. 

•  New  model  classes:  Source,  link,  network,  and  closed  loop. 

•  New  problems:  Active,  multi-route,  and  high  resolution  measurements. 

Stimulus  to  Theory 

•  Statistical  estimation:  For  time  series  with  infinite  moments  and/or  scaling. 

•  Queueing  Theory:  Dealing  with  sub-exponential  (eg  LRD)  input  processes. 

•  Applied  Probability:  Convergence  results  for  stochastic  processes. 

•  Algorithms:  On  and  Off-line  synthesis  methods  for  very  long  time  series. 

•  Analysis:  Network  feedback,  and  properties  of  new  models. 
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Tele-Traffic  Today 


Measurement  Practice 

•  Recognition  of  measurement,  not  just  routine  quantification,  but  discovery. 

•  Widespread  monitoring  of  LAN’s,  ISP  networks,  national  infrastructures . 

•  Large  scale  programs,  from  routine  monitoring  to  ultra  high  resolution. 

•  Emergence  of  numerous  small  &  large  scale  active  probing  efforts. 

•  Huge  advances  in  measurement  accuracy. 

•  Router  based  measurements  widely  used. 
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Tele-Traffic  Today 


Renaissance  in  Modelling 

•  Return  of  science  101 :  Observation  and  inspiration  before  model.. 

•  Return  of  verification:  Evaluating  usefulness  against  real  data. 

•  New  model  paradigm  as  standard:  Characteristic  scale  — ►  scale  invariance. 

•  New  model  classes:  Source,  Link,  and  Network. 

•  New  problems:  Active,  Multi-route,  and  High  resolution  measurements. 
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Tele-Traffic  Today 


Stimulus  to  Theory 

•  Statistical  estimation:  For  time  series  with  infinite  moments  and/or  scaling. 

•  Queueing  Theory:  Dealing  with  sub-exponential  (eg  LRD)  input  processes. 

•  Applied  Probability:  Convergence  results  for  stochastic  processes. 

•  Algorithms:  On  &  Off-line  synthesis/analysis  for  long  time  series. 

•  Analysis:  Properties  of  new  models,  especially  mixed  open  and  closed  loop. 
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Passive  and  Active  Measurement 


Typical  Passive  Aims 

•  “At-a-point”  or  “Network  core”. 

•  Backbone  link  utilisation,  Link  traffic  patterns,  Server  workloads. 

•  Long  term  monitoring:  Dimensioning,  Capacity  Planning,  Source  modelling. 

•  Engineering  view:  Network  performance. 

Typical  Active  Aims 

•  “End-to-End”  or  “Network  edge”. 

•  End-to-End  Loss,  Delay,  Connectivity,  “Discovery”  and  “Tomography”. 

•  Long  and  short  term  monitoring,  “Network  health”  and  “Route  state”. 

•  Internet  view:  Application  performance  and  robustness. 
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Major  Measurement  Programs 


•  9 
* 
* 

9 

9 

* 

* 

•  9 

•  ♦ 
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Passive  Active  Tools 

NLANR  (National  Laboratory  for  Applied  Network  Research). 
CAIDA  (Cooperative  Association  for  Internet  Data  Analysis). 
WAND  (Waikato  Applied  Network  Dynamics  [DAG  cards]). 

PingER  (Ping  End-to-end  Reporting). 

Surveyor  (From  Advanced  Networks  and  Services). 

RIPE— NCC  (Network  Coordination  Centre). 

NIMI  (National  Internet  Measurement  Infrastructure). 

4k  AT&T  NetScope. 

Cicso  Netflow. 

NetraMet. 
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skitter 

AS  internet  graph 


Peering: 

OutDegree 


association  tor  internet  data  analysis  O  san  dleao  supercomputer  center  Q  university  of  California,  san  dieao 


Tracing  Network  Nodes 


Reproduced  with  permission  of  CAIDA,  (6)  2001  CAIDA/UC  Regents.  Mapnet  Author:  Bradley  Huffaker. 
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Flows  and  Packets 


Flows  are  sets  of  packets  associated  to  the  same  data  transfer. 
Flow  arrivals:  Y(t) 


Packet  arrivals:  X(t) 


Time 


Sequence  Number 


Zooming  on  a  Single  Link 


17 


Sequence  Number 
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260 
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Sequence  Number 


260 


260.5 


262 


Traffic  Data  — ►  Timeseries 


What  are  we  Measuring? 

•  Internet  Protocol  (IP)  packets,  the  unit  of  transport  across  networks. 

•  Data  split  into  packets,  with:  header,  payload. 

•  Payload  carries  higher  layer  protocols:  TCP,  UDP,  ICMP. 

•  Protocols  support  services  &  applications,  more  protocols: 

•  TCP:  HTTP,  FTP,  SNMP,  ...  (for  reliable  data) 

•  UDP:  VoIP,  DNS,  NTP„..  (for  real  time) 

•  For  each  packet: 

•  Could  filter  based  on  criteria  (address,  type,  size, ...) 

•  Could  capture  all  or  part  (e.g.  just  the  header). 

•  Must  timestamp. 

•  Key  concept,  a  flow  (collection)  of  packets. 


Two  Point  Processes  from  Traffic 


Flow  arrivals:  Y ( t ) 


Time 

Packet  arrivals:  X  (t) 

Most  common  time  series  extracted  are  packets  or  bytes  per  bin 


Prelude  to  a  Paradigm 


Seeing  Packet  Traffic 


•  1991:  ISDN  (Hellstern,  Wirth,  Yan,  Hoeflin) 

•  1991 :  Ethernet  (Leyland,  Wilson) 

•  1993:  Ethernet  (Erramilli,  Willinger) 


-  infinite  moments. 
-  bursts  at  all  time  scales. 

-  fractal  properties. 
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The  Arrival  of  Fractal  Traffic 


Believing  Packet  Traffic 

•  1993:  Ethernet  (Leyland,  Taqqu,  Willinger,  Wilson)  -  Self-Similar  Traffic. 

•  1994:  CCSN/SS7  (Duffy,  McIntosh,  Rosenstein,  Willinger)  -Self-Similar. 

•  1994:  Internet  (Paxson,  Floyd)  -  Failure  of  Poisson  modelling. 

•  1994  —»••••  LAN’s  across  the  world  see  -Self-Similarity. 

•  1994:  Web  docs  (Cunha,  Bestavros,  Crovella)  -  Heavy  tailed  file  sizes. 

•  1995:  Video  (Beran,  Sherman,  Taqqu,  Willinger)  -  roughly  Self-Similar. 
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The  Self-Similarity  of  Ethernet  Traffic 


The  reference  Bellcore  trace,  ‘pAug’,  is  close  to  Fractional  Gaussian  Noise. 
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Measuring  the  Exponent  using  Wavelets 


Wavelet  coefficients  of  our  traffic  process: 


dx(j,k)  —  (xAj,k)- 

Spectral  definition  of  LRD  is 


~  cf\v\  a,  \v\  —*■  0,  with  a  G  (0, 1). 

In  this  case,  provided  N  >  1  vanishing  moments: 

E\dx(j,  k)  |2  ~  CfC(a)  2aj ,  j  +oo 

Estimating  the  LHS  from  data  using 


s2(j)  —  — J2\dxti,k)\2, 


nJ  k 


where  nj  is  the  number  of  dx (j,  k)  available  at  octave  j  (scale  a  —  2i), 


We  use  ‘Logscale  Diagram’  to  refer  to  the  log-log  plot: 

LD:  log2520)  vs  j, 

in  which  straight  lines  are  evidence  for  scaling,  slope=  a. 


NATIONAL 

ICT  AUSTRALIA 

LIMITED 


o 


Analysis  of  Trace  ‘pAug’ 


ETHERNET:  bytes  per  12ms  bin. 

Logscale  Diagram,  N=2  [  (j1  ,j2)=  (3, 1 5),  oc-est  =  0.59,  Q=  0.01 1384],  D-init 


An  Old  Story  for  Science  and  Nature 


What  does  it  mean? 

•  No  characteristic  time-scale  controlling  the  dynamics  /  statistics. 

•  Statistically,  All  scales  (in  a  range)  are  equivalent,  under  a  renormalisation. 

•  Radical  temporal  burstiness:  no  natural  burst  scale. 

•  Key  parameters  no  longer  special  scales,  but  relations  across  scales. 

•  Absolute  quantities  — ►  scale  dependent  quantities. 

•  Affects  system  description,  behaviour,  measurement .... 
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The  Self-Similar  Traffic  Model 


Fractional  Gaussian  Noise  (fGn)  and  Fractional  Brownian  Motion  (fBm) 


The  unique  Gaussian  processes  which  are 


Stationary;  Cov  [Xfj]  (A:) 
Stationary  Increments;  Var  [Zj-j(k)] 


1)2H  +  2k2H  +  (fe  +  1)2H 


Corresponding  traffic  models: 

Rate:  R(t ) 
Total  Traffic:  W (t) 


M  +  crXH{t) 
fit  +  aZH(t) 


where 


t 


zH(t)  —  "52  X I {(i), 
i=  1 


W(t) 


=  E«(0 


i=  1 
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The  Long-Range  Dependent  (LRD)  Traffic  Models 


LRD  definition:  a  slowly  decaying  covariance 

r x(k)  ~  Crk-P,  0  <  p  <  1, 

where  (p  =  2-  2  h). 

Corresponding  traffic  rate  model: 

R(t)  =  fi  +  aXpCrk*(t). 


LRD  more  general  than  H-SS: 


•  Second  order  description  only,  not  necessarily  Gaussian! 

•  Has  been  called  second  order  asymptotically  self-similar  (but  careful!) 

•  Heavy  tail  ‘begins’  only  after  some  cutoff  scale  k*. 

•  Tail  may  be  ‘thin’,  low  mass  (small  cr),  independent  of  variance! 

•  Small  scale  structure  unspecified. 

•  At  a  minimum,  three  correlation  parameters,  not  just  H. 
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Non-Gaussian  LRD  -  the  On/Off  Source 


•  Alternating  renewal  process:  {Aj}  i.i.d.  {Bj}  i.i.d. 

•  LRD  if  A  or  B  heavy  tailed: 

•  If  Pr (B  >  x)  ~  cx~a,  /3lrd  =  3 -a,  cr  — 

•  Efficient  generation  (0(1)  computation  and  state) 

Corresponding  traffic  rate  models: 

•  as  active/silence  sources. 

•  as  building  blocks  for  a  compound  source,  eg. 

•  N  t,  p  =  Pr(On)  — *•  0,  A  =  const,  h  =const:  — >  M/G/oo 

•  N  t,  p  =const,  A  =  const,  h  — >  0:  — >  fGn 
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Impacts  on  Traffic  Modelling 


Statistics 

•  Difficult  estimation  of  almost  everything. 

•  Difficult  model  choice:  Non-stationarity  confusions  and  others. 

•  Large  confidence  intervals  (need  more  data). 

•  Wavelets  can  help  kill  LRD,  mitigate  NS,  in  many  cases. 
Complexity  only  O(n)  and  can  be  done  on-line. 


Queueing 

•  Larger  buffer  sizes,  lower  utilisation  at  given  QoS. 


•  ‘Buffer  insensitivity’  (bigger  won’t  help  much). 

•  Impact  depends  a  lot  on  the  amplitude  burstiness: 

•  fBm  storage  (Norros)  -  Weibullian  tail. 

•  large  peak  On/Off  superpositions  -  Regularly  varying  tails,  infinite  mean 
(Cohen;  Boxma;  Whitt). 


Service  discipline  is  important  -  can  reinstate  finite  mean  (Boxma  et  al.) 
Critical  time  scales  can  exist,  but  value  depends  on  full  structure. 
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Lessons  for  Network  Performance 


•  Heavy  tails  can  be  thinned,  but  not  cut  off. 

•  Smoothing  the  beast  will  not  tame  it. 

•  Statistical  Multiplexing  works  as  ever  -  recommended. 

•  Processor  Sharing  helps  the  individual,  but  not  the  society. 

•  Hi  Peak  rates  can  kill. 

•  Power-laws  are  very  persistent,  if  you  can’t  kill  it: 

Make  sure  it  is  irrelevant,  or 

Make  it  work  for  you,  or 

Understand  it,  calm  it  down,  then  live  with  it. 

•  LRD  not  everything,  obviously:  variance,  marginal. 
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BUT:  Internet,  TCP,  Small  Scales,  and  Non-Gaussianity 


The  ‘Self-Similarity’  model  has  limits 

•  Only  really  asymptotic:  true  for  ‘large  scales’,  ss>  1  second. 

•  Really  requires  Gaussianity  -  often  far  from  the  case! 

•  Range  of  scales  where  valid  may  not  be  crucial  for  engineering  purposes. 

•  At  small  scales,  feedback  can  mold  the  beast. 
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High  Amplitude  Burstiness  at  Small  Scale 


x  10' 


03 

> 


c 

o 

-*—• 

o 

CD 

c 

c 

o 

O 


1000 

bin  #  (40ms) 


2000 


So  small  scales  are  much  harder ... 
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Examples  of  Scaling  in  Traffic:  2nd  Order  Wavelet  Analysis 


ETHERNET:  bytes  per  12ms  bin. 


Logscale  Diagram,  N=2  [  (j1J2)=  (3,15),  a-est  =  0.59,  Q=  0.01 1384],  D-init 


INTERNET:  new  connections  per  10ms  bin. 

Logscale  Diagram,  N=2  [  (j1  ,j2)=  (8,1 9),  a-est  =  0.59,  Q=  0.81665] 
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UDP 


Biscaling  in  TCP  Arrivals 

All  Connections  TCP 


Collaboration  with  Patrice  Abry  and  Nicolas  Hohn 
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Evidence  for  Multifractality 


Wavelet  gth  order  moments:  E| dx(j,  k) \q  C  2a<iJ ,  j  — »■  0. 


Estimating  the  LHS  from  data  using 


=  — J2\dx(j,k)\q, 


nJ  k 


and  measure  the  slopes  aq  =  C(qO  +  g/2  in  a  log-log  plot  (the  ‘MD‘). 


Instead  of  testing  for  linearity  of  C(g)  vs  q,  look  for  C (q)/q  vs  q  constant. 
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A  Short  History  of  Multifractal  Traffic  Modelling 


•  1997:  Ethernet  (Taqqu,  Teverovsky,  Willinger)  -  not  MF 

time  domain Idiscrete  packet  and  byte  counts! 5  =  10ms. 

•  1997:  TCP,  LAN  gateway  (Riedi,  Vehel)  -  MF  at ‘high  freq’ 

increments  I  packet  sizes,  iat’s,  packet  and  byte  counts!  S  =  150ms. 

•  1998:  Ethernet  (Abry,  Veitch)  -  not  MF 

wavelet  distributions! continuous  time  byte  counts. 

•  1997-8:  TCP,  LAN  gateway  (Feldmann,  Gilbert,  Willinger,  Kurtz)  -  MF  small,  Mono  large, 
wavelet  domain;  discrete  packet  counts! S  =  10ms  | large  &  small  regimes 

•  1998:  TCP,  WAN  (Feldmann,  Gilbert,  Willinger)  -  MF  at  small,  Mono  large 

wavelet  domain  (discrete  packet  counts! 5  =  10ms. 

•  1999:  ISP  and  simulated  (Feldmann,  Gilbert,  Pluang,  Willinger)  -  MF  at  small,  but  dirty 
wavelet  domain  (discrete  packet  counts! S  =  10ms. 

•  1999-2000:  TCP,  WAN  (Veitch,  Abry,  Flandrin,  Chainais)  -  IDC  holds  over  large  and  small, 
wavelet  distributions  (TCP  connection  counts!  S  =  10ms  Id’s  used. 

•  2001 :  TCP,  WAN  (Roux,  Veitch,  Abry,  Pluang,  Flandrin,  Micheel)  -  IDC,  but  ^mono. 
wavelet  distributions  (TCP  connection  counts,  packet  iar’s  and  counts,  byte  counts  S  = 
10ms  Id’s  used 

•  2003:  TCP,  WAN,  high  rate  (Zhang,  Ribeiro,  Moon,  Diot)  -  Mono  everywhere 

wavelet  domain  (discrete  byte  counts! S  =  10ms.  | Cl’s  used 

•  2003:  TCP,  WAN,  high  rate  (Hohn,  Veitch,  Abry)  -  Empirical  scaling  misleading? 

wavelet  domain  (continuous  packet  counts! <5  =  5 fis  — ►  5 ms  (Cl’s  used  \\q  =  2 


Similar  Data,  A  Less  Flattering  View 


Semi-Experiments  on  Packet  Arrivals 


i 


Time 


DATA  EXPERIMENT 


Original  TCP  Data 


Time 
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Permutation  of  Flows  [A-Perm] 


Time 


Permutation  of  Flows  [A-Perm] 


DATA  .  EXPERIMENT 


Original  Order  Poisson  Arrivals  [A-Pord] 


(a  smooth  form  of  internal  shuffling) 


Time 


DATA  .  EXPERIMENT 


Original  Order  Poisson  Arrivals  [A-Pord] 


Time 


Original  Order  Poisson  Arrivals  [A-Pord] 
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Permuted  Poisson  Arrivals  [A-Pois] 


Time 


Permuted  Poisson  Arrivals  [A-Pois] 


From  simple  (Semi-)Experiments,  we  learn  a  lot 


From  these  flow  arrival  manipulations : 

•  Correlations  between  flows  can  be  neglected 

•  No  need  for  session  level  hierarchical  models 

•  TCP  dynamics  between  flows  can  be  neglected 

•  For  IP  modelling,  flow  arrivals  can  be  modelled  as  Poisson 

•  Justifies  an  assumption  commonly  used  in  traffic  modelling. 

Note:  true  flow  arrival  process  is  LRD. 
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Semi-Experiment  Outcomes 


Flow  Arrival  Manipulation 

•  Correlation  between  flows  can  be  neglected, 

•  For  IP  modelling,  flow  arrivals  can  be  modelled  as  Poisson, 

•  [A-Clus]:  knee  of  Y  affects  knee  of  X! 

Packets  within  Flow  Manipulation 

•  LRD  not  caused  by  arrival  process  of  packets  within  flows, 

•  Small  scale  behaviour  governed  by  structure  within  flows, 

•  Finite  Poisson  process  a  reasonable  0-th  order  model, 

•  [P-Pois]:  indistinguishable  from  [P-Uni], 

•  [P-ConstR];  [P-ScaledR]:  rate  acts  like  a  scale  parameter, 

Flow  Selection 

•  Observed  LRD  caused  by  heavy-tailed  flow  volumes, 

•  [S-Thin]:  random  thinning  consistent  with  independent  flows, 

•  [S-Dur] :  also  kills  LRD  (flow  duration  slaved  to  volume), 

•  [S-Pkt] :  LRD  still  present  even  without  heavy  tail! 

Flow  Truncation  Manipulation 

•  [T-Pkt]:  also  kills  LRD,  makes  X  tend  to  Y, 
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Poisson  (Barlett-Lewis)  Cluster  Processes 


Definition 

•  A  Poisson  process  of  seeds  (flows),  initiating  independent  groups  of  points  (packets): 

X{t)  =  ^Qi{t  -  tF(i )) 

i 

•  Group:  a  finite  renewal  process  with  P  points  and  inter-arrival  distribution  A: 

p(0  j-i 

ddt)  =  J2s{t-J2A^) 

3=1  i=i 

Parameters 

•  Flow  arrivals:  constant  intensity  AF 

•  Flow  structure: 

•  Packet  arrivals:  A,  =  Aa<oo, 

•  Flow  volume:  P,  EP  =  ^P<oo, 


cf:  04(cj),o;>O 
Pgf:  GP(z)  =  Ejlo PjzJ>  M  <  1- 


Fourier  Spectrum 


rxO)  =  A  F(^n,(i/)  +  (S(,(w)  +  S(,(-w))), 
re(»:  spectrum  of  stationary  renewal  process  with  inter-arrivals  A,  and 

K(Se(u))  =  (Gp(<J>,(o>))  -  l). 

(i-04O))2\  / 


Verify  LRD: 

Se(o>)  )  LB(/3)(27rAA)2-/Ja>-(2-/J)  -►  oo 

u>—>oc  COS(c7t/2) 

- - y  (J 

( mc 

where  B(/3)  =  ^(1  -  P)  cos(7r/3/2)/(27r)(2_/3)  >  0 


Properties 


•  Af  just  a  variance  multiplier:  ‘more  of  same’ 

•  has  scale  parameter  1/Aa  if  A  has:  AF,  Aa,  c,  Fp)  =  Tx(^/Aa;  Af,  1,  c,  FP) 

•  Two  terms  dominating  small-large  scales 

•  First  term  (small  scales):  scaled  renewal  process,  no  detailed  P  dependence 

•  Second  term  (large  scales):  LRD,  no  A  dependence  beyond  Aa 
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Modelling  a  Typical  Auckland  IV  Trace 


0.0039  0.016  0.062  0.25  1  4  16  64  256  1024 


j  =  log2  ( a ) 


*:  Jgr  =  -  log2  Aa  +  log2(7r2(c  +  l)/(3ec2)) 

D:  3 pgr  =  -log2AA  +  ^-—  (log2/tP-  log2(2LB(/3))  -  log2c) 


=  C/q 


Comparison  between  data  and  fitted  Cluster  Model 


Order  q  Order  q 


